UNCLASSIFIED


Figure 1: Space debris orbiting Earth.
Image from MIT news website depicting the large volume of space debris orbiting the Earth at ~17,500 mph.



Executive Summary

  • Charged with informing the process to develop a debris on debris collision warning threshold to mitigate future collision risk and limit false alarms.

  • The study has focused on the following parameters:
    • Debris collision results in at least one fragment at least five centimeters in diameter.
    • Five days to time of closest approach (TCA).
    • Concern Events terminology defined as \(\text{Collision Probability} >= 1\text{e-}5\) within a day to TCA.

  • Final study implementations:
    • Removed all observations with \(<= 1\text{e-}10\text{ Collision Probability}\).
    • Tested

Problem Formulation

  • Problem Statement: Under what circumstances should debris-on-debris collision warnings be used?
    • What probability of collision should be used as a surrogate for an actual event of concern?
    • What probability of collision should trigger a warning at five days to TCA?
    • How well can we do in warning for events occurring in the future?

  • Mission Statement: Recommend debris collision notification thresholds and develop a mechanism to explore the notification decision space to assess risk of debris-on-debris collisions in space.

Exploratory Data Analysis

  • Questions:
    • How is the Collision Probability distributed?
    • How does the rate of new concern events change over time?
    • How does the Collision Probability change over time?

  • Data Preparation:
    • Summarize by each event to show the Collision Probability for each fragment size at the last recorded time, if that time is within a day of TCA.
    • Summarize by each event to show the Collision Probability for each fragment size at the last recorded time, for each binned days to TCA.

Collision Probability Density

We need to find a range of values to appropriately label events of concern.


Figure 2: Collision Probability within One Day of TCA



We use,

\(\text{Collision Probabilty} >= 1\text{e-}5 \text{ and days to TCA} < 1 \text{ day,}\)


as a surrogate to label a concern event.


Collision Probability Variability

  • Does the \(\text{Collision Probability}\) increase as we approach TCA?
  • Should we expect to have a lower warning threshold at five days to TCA?


Figure 4: Collision Probability as TCA Varies



Conclusions:

  • The variability expands slightly, but there is not overwhelming evidence of an increase in the \(\text{Collision Probabilty}\).
  • We consider the following range for \(\text{Collision Probability}\) warning thresholds:


\(1.94\text{e-}22 <= Pc\_warn <= 1\text{e-}5\),


where \(Pc\_warn\) is the \(\text{Collision Probability}\) that triggers a warning.


Risk Tradespace

We need to evaluate warning thresholds by examining the trade space between risk aversion and tolerance.

We have the following working definitions:


\(\text{Concern Event} := \text{Collision Probability} >= 1\text{e-}5 \text{ and days to TCA < 1 day}\),

\(\text{False Negative (FN)} := \text{Number of Concern Events that did not trigger a warning at 5 days to TCA}\), and

\(\text{False Positive (FP)} := \text{Number of false alarms, where warned events at 5 days to TCA did not become Concern Events}\).


We can now explore how the warning threshold effects the \(\text{FP}\) and \(\text{FN}\).


Figure 5: Confusion Rates at Five Days to TCA and Concern Prop = 1e-5


Five days to TCA

Hovering over the plot, we see that the warning threshold decreases as FN decreases and as FP increases. In this case, the more severe mistake is FN (not warning for a concern event), which is our Type II error. The FP (false alarm, Type I error) has much lower rates because of the high number of events with Collision Probabilities falling below \(1\text{e-}5\) within one day to TCA.

We recommend this chart to inform warning thresholds for five days to TCA, for concern events previously defined.

Five and four days to TCA

Five, four, and three days to TCA

Risk Tradespace Short Notice

Four days to TCA

Three days to TCA

Two days to TCA

Way Forward

  • Finalize warning threshold point estimate formulation.
  • Employ Monte Carlo method to explore the variability of the point estimate, and deliver a \(95% \text{bootstrap confidence inverval}\) for warning threshold at five days to TCA (Efron 1979).
  • Repeat this methodology for 4, 3, 2 days to TCA.
  • Iterate on the R Shiny Application based on client feedback.

Works Cited

Bengtsson, Henrik. 2019. “R.utils: Various Programming Utilities.” https://CRAN.R-project.org/package=R.utils.
Efron, B. 1979. Bootstrap Methods: Another Look at the Jackknife.” The Annals of Statistics 7 (1): 1–26. https://doi.org/10.1214/aos/1176344552.
Ren, Kun, and Kenton Russell. 2016. “Formattable: CreateFormattableData Structures.” https://CRAN.R-project.org/package=formattable.
RStudio, and Inc. 2019. “Htmltools: Tools for HTML.” https://CRAN.R-project.org/package=htmltools.
Sievert, Carson, Chris Parmer, Toby Hocking, Scott Chamberlain, Karthik Ram, Marianne Corvellec, Pedro Despouy, and Plotly Technologies Inc. 2020. “Plotly: Create Interactive Web Graphics via ’Plotly.js’.” https://CRAN.R-project.org/package=plotly.
Spinu, Vitalie, Garrett Grolemund, Hadley Wickham, Ian Lyttle, Imanuel Constigan, Jason Law, Doug Mitarotonda, Joseph Larmarange, Jonathan Boiser, and Chel Hee Lee. 2020. “Lubridate: Make Dealing with Dates a Little Easier.” https://CRAN.R-project.org/package=lubridate.
Vaidyanathan, Ramnath, Yihui Xie, J. J. Allaire, Joe Cheng, Kenton Russell, and RStudio. 2019. “Htmlwidgets: HTML Widgets for R.” https://CRAN.R-project.org/package=htmlwidgets.
Wickham, Hadley, Romain François, Lionel Henry, Kirill Müller, and RStudio. 2020. “Dplyr: A Grammar of Data Manipulation.” https://CRAN.R-project.org/package=dplyr.
Xie, Yihui, Joe Cheng, Xianying Tan, J. J. Allaire, Maximilian Girlich, Greg Freedman Ellis, Johannes Rauh, et al. 2020. DT: A Wrapper of the JavaScript LibraryDataTables’.” https://CRAN.R-project.org/package=DT.
UNCLASSIFIED